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Final  Report 

Auctions  and  similar  market-based  mechanisms  are  decentralized  methods  that  are  a  promising  way  for 
teams  of  agents  to  assign  and  re-assign  tasks  among  themselves  in  dynamic,  partially  known  and  time- 
constrained  domains.  In  this  case,  the  agents  are  the  bidders  and  the  tasks  are  the  goods  up  for  auction. 
Every  agent  then  executes  the  tasks  that  it  wins.  As  the  agents  discover  more  about  the  environment 
during  execution  or  as  the  environment  or  tasks  change,  they  run  additional  auctions  to  re-optimize  the 
allocation  of  tasks  among  themselves.  Auction-based  coordination  systems  promise  to  be  efficient  in 
terms  of  communication  (agents  communicate  only  essential  information,  such  as  numeric  bids  which 
encapsulate  large  amounts  of  local  information)  and  computation  (agents  compute  their  bids  in 
parallel). 

Our  project  was  an  interdisciplinary  effort  of  two  researchers  who  have  already  worked  with  each  other 
productively  on  other  topics.  One  of  the  Pis  was  a  researcher  in  artificial  intelligence  from  the  University 
of  Southern  California,  and  the  other  PI  was  a  researcher  in  theoretical  computer  science  and  operations 
research  from  Georgia  Institute  of  Technology,  which  enables  us  to  build  systems  with  very  good 
performance  on  top  of  sound  mathematical  foundations.  In  particular,  we  designed  efficient  and 
effective  auction  mechanisms  for  the  cooperative  real-time  allocation  of  tasks  and  applied  the  results  to 
simulations  of  mine  clearing  tasks  and  search-and-rescue  tasks.  In  general,  auction-based  coordination 
systems  based  on  sequential  single-item  auctions  result  in  no  more  bids  and  provide  much  better 
performance  guarantees  than  auction-based  coordination  systems  based  on  parallel  single-item 
auctions.  They  are  also  much  easier  to  implement  than  auction-based  coordination  systems  based  on 
combinatorial  auctions  (since  the  number  of  bids  is  much  smaller  and  winner  determination  is  much 
simpler  and  faster)  and  allow  for  a  decentralized  implementation.  We  therefore  designed  efficient  and 
effective  auction  mechanisms  for  the  cooperative  real-time  allocation  of  tasks  based  on  sequential 
single-item  auctions  and  improved  them  in  several  ways,  bringing  to  bear  insights  from  different 
disciplines,  namely  robotics  and  artificial  intelligence  on  one  hand  and  theoretical  computer  science  and 
operations  research  on  the  other  hand.  We  also  analyzed  their  properties  theoretically  and  evaluated 
them  experimentally  to  characterize  when  they  should  be  used  and  how  good  they  are,  creating  a 
mathematical  framework  for  the  design  and  analysis  of  auctions  for  the  cooperative  real-time  allocation 
of  tasks. 

While  there  is  interest  in  the  research  community  in  using  auctions  for  the  coordination  of  teams  of 
agents,  most  of  the  work  had  been  empirical  and  somewhat  ad-hoc  in  nature.  Auctions  have,  of  course, 
long  been  studied  in  economics.  However,  the  auction  literature  in  economics  assumes  a  competitive 
context  with  rational,  and  hence  long,  decision  cycles  rather  than  a  cooperative  context  with  extreme 
time  pressure.  Thus,  while  some  of  its  insights  could  be  exploited  for  our  purposes  (for  example,  the 
concept  of  synergy),  most  of  them  did  not  apply  (for  example,  incentive  compatibility  to  elicit  sincere 
bids)  since  one  controls  both  the  auction  mechanism  and  the  preference  structures  of  the  agents. 

It  is  often  crucial  to  coordinate  teams  of  cooperating  agents  well.  Auction-based  coordination  applies  to 
a  wide  range  of  real-time  domains  with  multiple  agents  and  tasks,  where  there  are  positive  and  negative 
synergies  among  tasks,  including  on-line  distributed  routing,  role  assignment,  scheduling  or  control.  The 
simplest  scenario  that  we  studied  was  on-line  distributed  routing,  where  a  set  of  targets  is  given  and 
each  one  of  them  needs  to  be  visited  by  one  robot.  On-line  distributed  routing  includes  assigning 
accidents  to  ambulances,  incidents  to  police  cars,  customers  to  taxi  cabs,  rocks  to  Mars  rovers  (to  take 


and  analyze  rock  probes),  observation  locations  to  robots  that  map  terrain  or  wings  of  an  art  gallery  to 
guard  robots.  Thus,  it  applies  directly  to  tasks  such  as  mine  clearing  and  search-and-rescue,  which  we 
used  as  both  motivating  scenarios  and  testbeds.  Our  experimental  results  show  that  it  is  feasible  to 
allocate  hundreds  of  tasks  in  real-time.  We  also  performed  experiments  in  an  existing  search-and-rescue 
simulation  to  be  relevant  to  applications. 

Overall,  our  research  showed  that  auctions  appear  to  be  well  suited  for  the  cooperative  real-time 
allocation  of  tasks  and  can  be  used  as  foundation  for  building  coordination  systems.  There  now  exists 
the  beginning  of  a  theoretical  framework  for  understanding  their  properties  and  for  extending  their 
capabilities  to  more  realistic  settings.  Our  findings  are  summarized  in  our  publications,  listed  below.  We 
provide  the  abstracts  of  some  key  publications  to  give  the  reader  an  overview  of  our  main  results  and 
insights.  We  have  also  attached  a  copy  of  our  paper 

S.  Koenig,  P.  Keskinocak  and  C.  Tovey.  Progress  on  Agent  Coordination  with  Cooperative  Auctions 
[Senior  Member  Paper],  In  Proceedings  of  the  AAAI  Conference  on  Artificial  Intelligence  (AAAI),  2010. 

which  gives  a  good  overview  of  our  main  results  and  insights  up  to  and  including  2010. 

We  also  studied  pursuit-evasion  problems  where  a  number  of  pursuers  have  to  clear  a  given  graph  of 
potentially  fast-moving  evaders  despite  poor  visibility,  for  example,  where  robots  search  a  cave  system 
to  find  criminals.  We  studied  when  polynomial-time  algorithms  exist  to  determine  how  many  robots  are 
needed  to  clear  a  given  graph  and  how  a  given  number  of  robots  should  move  on  the  graph  to  clear  it 
with  either  a  minimum  sum  of  their  travel  distances  or  minimum  task-completion  time. 


Publications 


In  the  following,  we  give  the  abstracts  of  some  of  our  key  publications.  These  publications  acknowledge 
ARO  funding  as  well  as  funding  from  other  grants.  The  full  text  of  most  of  these  publications  is  available 
at  idm-lab.org/publications.html. 


2012 

S.  Koenig.  Making  Good  Decisions  Quickly.  The  IEEE  Intelligent  Informatics  Bulletin,  13,  (1),  14-20,  2012. 


Abstract:  Several  disciplines,  including  artificial  intelligence,  operations  research  and  many  others,  study 
how  to  make  good  decisions.  In  this  overview  article,  we  argue  that  the  key  to  making  progress  in  our 
research  area  is  to  combine  their  ideas,  which  often  requires  serious  technical  advances  to  reconcile 
their  different  assumptions  and  methods  in  a  way  that  results  in  synergy  among  them.  To  illustrate  this 
point,  we  give  a  broad  overview  of  our  ongoing  research  on  search  and  planning  (with  a  large  number  of 
students  and  colleagues,  both  at  the  University  of  Southern  California  and  elsewhere)  to  demonstrate 
how  to  combine  ideas  from  different  decision  making  disciplines.  For  example,  we  describe  how  to 
combine  ideas  from  artificial  intelligence,  operations  research,  and  utility  theory  to  create  the 
foundations  for  building  decision  support  systems  that  fit  the  risk  preferences  of  human  decision  makers 
in  high-stake  one-shot  decision  situations  better  than  current  systems.  We  also  describe  how  to 
combine  ideas  from  artificial  intelligence,  economics,  theoretical  computer  science  and  operations 


research  to  build  teams  of  robots  that  use  auctions  to  distribute  tasks  autonomously  among  themselves, 
and  give  several  more  examples. 

2011 

R.  Borie,  C.  Tovey  and  S.  Koenig.  Algorithms  and  Complexity  Results  for  Graph-Based  Pursuit  Evasion. 
Autonomous  Robots,  31,  (4),  317-332,  2011. 

Abstract:  We  study  the  classical  edge-searching  pursuit-evasion  problem  where  a  number  of  pursuers 
have  to  clear  a  given  graph  of  fast-moving  evaders  despite  poor  visibility,  for  example,  where  robots 
search  a  cave  system  to  ensure  that  no  terrorists  are  hiding  in  it.  We  study  when  polynomial-time 
algorithms  exist  to  determine  how  many  robots  are  needed  to  clear  a  given  graph  (minimum  robot 
problem)  and  how  a  given  number  of  robots  should  move  on  the  graph  to  clear  it  with  either  a  minimum 
sum  of  their  travel  distances  (minimum  distance  problem)  or  minimum  task-completion  time  (minimum 
time  problem).  The  robots  cannot  clear  a  graph  if  the  vertex  connectivity  of  some  part  of  the  graph 
exceeds  the  number  of  robots.  Researchers  therefore  focus  on  graphs  whose  subgraphs  can  always  be 
cut  at  a  limited  number  of  vertices,  that  is,  graphs  of  low  treewidth,  typically  trees.  We  describe  an 
optimal  polynomial-time  algorithm,  called  CLEARTHETREE,  that  is  shorter  and  algorithmically  simpler 
than  the  state-of-the-art  algorithm  for  the  minimum  robot  problem  on  unit-width  unit-length  trees.  We 
then  generalize  prior  research  to  both  unit-width  arbitrary-length  and  unit-length  arbitrary-width  graphs 
and  derive  both  algorithms  and  time  complexity  results  for  a  variety  of  graph  topologies.  Pursuit-evasion 
problems  on  the  former  graphs  are  generally  simpler  than  pursuit-evasion  problems  on  the  latter 
graphs.  For  example,  the  minimum  robot  and  distance  problems  are  solvable  in  polynomial  time  on  unit- 
width  arbitrary-length  trees  but  NP-hard  on  unit-length  arbitrary-width  trees. 

X.  Zheng  and  S.  Koenig.  Generalized  Reaction  Functions  for  Solving  Complex-Task  Allocation  Problems.  In 
Proceedings  of  the  International  Joint  Conference  on  Artificial  Intelligence  (IJCAI),  pages  478-483,  2011. 

Abstract:  We  study  distributed  task-allocation  problems  where  cooperative  agents  need  to  perform 
some  tasks  simultaneously.  Examples  are  multi-agent  routing  problems  where  several  agents  need  to 
visit  some  targets  simultaneously,  for  example,  to  move  obstacles  out  of  the  way  cooperatively.  In  this 
paper,  we  first  generalize  the  concept  of  reaction  functions  proposed  in  the  literature  to  characterize 
the  agent  costs  of  performing  multiple  complex  tasks.  Second,  we  show  how  agents  can  construct  and 
approximate  reaction  functions  in  a  distributed  way.  Third,  we  show  how  reaction  functions  can  be  used 
by  an  auction-like  algorithm  to  allocate  tasks  to  agents.  Finally,  we  show  empirically  that  the  team  costs 
of  our  algorithms  are  substantially  smaller  than  those  of  an  existing  state-of-the-art  allocation  algorithm 
for  complex  tasks. 

2010 

X.  Zheng  and  S.  Koenig.  Sequential  Incremental-Value  Auctions.  In  Proceedings  of  the  AAAI  Conference 
on  Artificial  Intelligence  (AAAI),  2010. 

Abstract:  We  study  the  distributed  allocation  of  tasks  to  cooperating  robots  in  real  time,  where  each 
task  has  to  be  assigned  to  exactly  one  robot  so  that  the  sum  of  the  latencies  of  all  tasks  is  as  small  as 
possible.  We  propose  a  new  auction-like  algorithm,  called  Sequential  Incremental-Value  (SIV)  auction, 
which  assigns  tasks  to  robots  in  multiple  rounds.  The  idea  behind  SIV  auctions  is  to  assign  as  many  tasks 


per  round  to  robots  as  possible  as  long  as  their  individual  costs  for  performing  these  tasks  are  at  most  a 
given  bound,  which  increases  exponentially  from  round  to  round.  Our  theoretical  results  show  that  the 
team  costs  of  SIV  auctions  are  at  most  a  constant  factor  larger  than  minimal. 

S.  Koenig,  P.  Keskinocak  and  C.  Tovey.  Progress  on  Agent  Coordination  with  Cooperative  Auctions 
[Senior  Member  Paper],  In  Proceedings  of  the  AAAI  Conference  on  Artificial  Intelligence  (AAAI),  2010. 

Abstract:  Auctions  are  promising  decentralized  methods  for  teams  of  agents  to  allocate  and  re-allocate 
tasks  among  themselves  in  dynamic,  partially  known  and  time-constrained  domains  with  positive  or 
negative  synergies  among  tasks.  Auction-based  coordination  systems  are  easy  to  understand,  simple  to 
implement  and  broadly  applicable.  They  promise  to  be  efficient  both  in  communication  (since  agents 
communicate  only  essential  summary  information)  and  in  computation  (since  agents  compute  their  bids 
in  parallel).  Artificial  intelligence  research  has  explored  auction-based  coordination  systems  since  the 
early  work  on  contract  networks  [Smithl980],  mostly  from  an  experimental  perspective.  This  overview 
paper  describes  our  recent  progress  towards  creating  a  framework  for  the  design  and  analysis  of 
cooperative  auctions  for  agent  coordination. 

K.  Daniel,  R.  Borie,  S.  Koenig  and  C.  Tovey.  ESP:  Pursuit  Evasion  on  Series-Parallel  Graphs  [Poster 
Abstract],  In  Proceedings  of  the  International  Joint  Conference  on  Autonomous  Agents  and  Multiagent 
Systems  (AAMAS),  2010. 

Abstract:  We  develop  a  heuristic  approach,  called  ESP,  that  solves  large  pursuit-evasion  problems  on 
series-parallel  (that  is,  treewidth-2)  graphs  quickly  and  with  small  costs.  It  exploits  their  topology  by 
performing  dynamic  programming  on  their  decomposition  graphs.  We  show  that  ESP  scales  up  to  much 
larger  graphs  than  a  strawman  approach  based  on  previous  results  from  the  literature. 

2009 

R.  Borie,  C.  Tovey  and  S.  Koenig.  Algorithms  and  Complexity  Results  for  Pursuit-Evasion  Problems.  In 
Proceedings  of  the  International  Joint  Conference  on  Artificial  Intelligence  (IJCAI),  pages  59-66,  2009. 

Abstract:  We  study  pursuit-evasion  problems  where  a  number  of  pursuers  have  to  clear  a  given  graph. 
We  study  when  polynomial-time  algorithms  exist  to  determine  how  many  pursuers  are  needed  to  clear 
a  given  graph  and  how  a  given  number  of  pursuers  should  move  on  the  graph  to  clear  it  with  either  a 
minimum  sum  of  their  travel  distances  or  minimum  task-completion  time.  We  generalize  prior  work  to 
both  unit-width  arbitrary-length  and  unit-length  arbitrary-width  graphs  and  derive  both  algorithms  and 
complexity  results  for  a  variety  of  graph  topologies.  In  this  context,  we  describe  a  polynomial-time 
algorithm,  called  CLEARTHETREE,  that  is  much  shorter  and  algorithmically  simpler  than  the  state-of-the- 
art  algorithm  for  the  minimum  pursuer  problem  on  trees.  Our  theoretical  research  lays  a  firm 
theoretical  foundation  for  pursuit  evasion  on  graphs  and  informs  practitioners  about  which  problems 
are  easy  and  which  ones  are  hard. 

X.  Zheng  and  S.  Koenig.  K-Swaps:  Cooperative  Negotiation  for  Solving  Task-Allocation  Problems.  In 
Proceedings  of  the  International  Joint  Conference  on  Artificial  Intelligence  (IJCAI),  pages  373-379,  2009. 

Abstract:  In  this  paper,  we  study  distributed  algorithms  for  cooperative  agents  that  allow  them  to 
exchange  their  assigned  tasks  in  order  to  reduce  their  team  cost.  We  define  a  new  type  of  contract, 


called  K-swaps,  that  describes  multiple  task  exchanges  among  multiple  agents  at  a  time,  which 
generalizes  the  concept  of  single  task  exchanges.  We  design  a  distributed  algorithm  that  constructs  all 
possible  K-swaps  that  reduce  the  team  cost  of  a  given  task  allocation  and  show  that  each  agent  typically 
only  needs  to  communicate  a  small  part  of  its  local  computation  results  to  the  other  agents.  We  then 
demonstrate  empirically  that  K-swaps  can  reduce  the  team  costs  of  several  existing  task-allocation 
algorithms  significantly  even  if  K  is  small. 

A.  Ekici,  P.  Keskinocak  and  S.  Koenig.  Multi-Robot  Routing  with  Linear  Decreasing  Rewards  over  Time.  In 
Proceedings  of  the  IEEE  International  Conference  on  Robotics  and  Automation  (ICRA),  pages  958-963, 
2009. 

Abstract:  We  study  multi-robot  routing  problems  (MR-LDR)  where  a  team  of  robots  has  to  visit  a  set  of 
given  targets  with  linear  decreasing  rewards  over  time,  such  as  required  for  the  delivery  of  goods  to 
rescue  sites  after  disasters.  The  objective  of  MR-LDR  is  to  find  an  assignment  of  targets  to  robots  and  a 
path  for  each  robot  that  maximizes  the  surplus,  which  is  defined  to  be  the  total  reward  collected  by  the 
team  minus  its  total  travel  cost.  We  develop  a  mixed  integer  program  that  solves  MR-LDR  optimally  with 
a  flow-type  formulation  and  can  be  solved  faster  than  the  standard  TSP-type  formulations  but  also  show 
that  solving  MR-LDR  optimally  is  NP-hard.  We  then  develop  an  auction-based  algorithm  and 
demonstrate  that  it  solves  MR-LDR  in  seconds  and  with  a  surplus  that  is  comparable  to  the  surplus 
found  by  the  mixed  integer  program  with  a  12  hour  time  limit. 

X.  Zheng  and  S.  Koenig.  Negotiation  with  Reaction  Functions  for  Solving  Complex  Task  Allocation 
Problems.  In  Proceedings  of  the  IEEE  International  Conference  on  Intelligent  Robots  and  Systems  (IROS), 
pages  4811-4816,  2009. 

Abstract:  We  study  task-allocation  problems  where  cooperative  robots  need  to  perform  tasks 
simultaneously.  We  develop  a  distributed  negotiation  procedure  that  allows  robots  to  find  all  task 
exchanges  that  reduce  the  team  cost  of  a  given  task  allocation,  without  robots  having  to  know  how 
other  robots  compute  their  robot  costs.  Finally,  we  demonstrate  empirically  that  our  negotiation 
procedure  can  substantially  reduce  the  team  costs  of  task  allocations  resulting  from  existing  task- 
allocation  procedures,  including  sequential  single-item  auctions. 
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Combinatorial  Search  (SoCS),  2012.  A  summary  of  this  paper  appeared  also  in:  C.  Hernandez,  J.  Baier,  T. 
Uras  and  S.  Koenig,  New  Developments  in  Real-Time  Heuristic  Search:  A  Demo  [System  Demonstration 
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Abstract 

Auctions  are  promising  decentralized  methods  for  teams  of 
agents  to  allocate  and  re-allocate  tasks  among  themselves 
in  dynamic,  partially  known  and  time-constrained  domains 
with  positive  or  negative  synergies  among  tasks.  Auction- 
based  coordination  systems  are  easy  to  understand,  sim¬ 
ple  to  implement  and  broadly  applicable.  They  promise  to 
be  efficient  both  in  communication  (since  agents  communi¬ 
cate  only  essential  summary  information)  and  in  computation 
(since  agents  compute  their  bids  in  parallel).  Artificial  intelli¬ 
gence  research  has  explored  auction-based  coordination  sys¬ 
tems  since  the  early  work  on  contract  networks  (Smith  1980), 
mostly  from  an  experimental  perspective.  This  overview  pa¬ 
per  describes  our  recent  progress  towards  creating  a  frame¬ 
work  for  the  design  and  analysis  of  cooperative  auctions  for 
agent  coordination. 

Introduction 

Centralized  control  is  often  inefficient  for  distributed  sys¬ 
tems  in  terms  of  both  the  required  amount  of  computation 
and  communication  since  the  central  controller  is  the  bottle¬ 
neck  of  the  system.  Many  researchers  have  therefore  studied 
agent  coordination  with  cooperative  auctions.  An  auction  is 
“a  market  institution  with  an  explicit  set  of  rules  determin¬ 
ing  resource  allocation  and  prices  on  the  basis  of  bids  from 
the  market  participants”  (McAfee  and  McMillan  1987).  In 
auction-based  coordination  systems,  the  bidders  are  agents, 
and  the  items  up  for  auction  are  tasks  to  be  executed  by  the 
agents.  All  agents  bid  their  costs.  Thus,  the  agent  with  the 
smallest  bid  cost  is  best  suited  for  a  task.  All  agents  then 
execute  the  tasks  that  they  win.  Economics  has  an  exten¬ 
sive  auction  literature  but  its  agents  are  rational  and  com- 

*  We  thank  Xiaoming  Zheng  for  his  overview  of  auction  applica¬ 
tions  and  contributions  to  the  development  of  algorithms  and  their 
experimental  evaluation.  This  material  is  based  on  some  of  our 
published  research  on  the  topic,  which  was  supported  by,  or  in  part 
by,  NSF  under  contract/ grant  number  0413196,  ARL/ARO  under 
contract/grant  number  W91  INF-08-1-0468  and  ONR  in  the  form 
of  a  MURI  under  contract/grant  number  N00014-09-1-1031.  The 
views  and  conclusions  contained  in  this  document  are  those  of  the 
authors  and  should  not  be  interpreted  as  representing  the  official 
policies,  either  expressed  or  implied,  of  the  sponsoring  organiza¬ 
tions,  agencies  or  the  U.S.  government. 
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petitive,  leading  to  long  decision  cycles,  strategic  behavior 
and  possibly  collusion.  Such  issues  do  not  arise  in  auction- 
based  coordination  systems  because  the  agents  simply  ex¬ 
ecute  their  program.  Thus,  while  some  insights  from  eco¬ 
nomics  can  be  exploited  for  building  auction-based  coordi¬ 
nation  systems  (for  example,  the  concept  of  synergy),  many 
of  them  do  not  apply.  Conversely,  auction-based  coordina¬ 
tion  systems  must  operate  in  real-time.  Some  researchers 
therefore  prefer  to  use  the  term  “auction-inspired  control  al¬ 
gorithms”  (for  decentralized  control)  over  “cooperative  auc¬ 
tions”  to  highlight  these  differences.  This  paper  provides  a 
unified  overview  of  our  progress  towards  creating  a  frame¬ 
work  for  the  design  and  analysis  of  cooperative  auctions  for 
agent  coordination,  drawing  from  publications  in  different 
venues,  including  robotics  and  agents  conferences. 

Applications 

Auction-based  coordination  systems  apply  to  a  wide  range 
of  real-time  domains. 

On-Line  Distributed  Role  Allocation 

Allocate  roles  to  agents  with  different  capabilities  for  the 
execution  of  a  given  plan  or  playbook  so  that  every  task 
is  performed  by  a  qualified  agent  (Hunsberger  and  Grosz 
2000).  Examples  include  allocating  attacker  and  defender 
roles  to  robots  in  RoboSoccer  (Frias-Martinez,  Sklar,  and 
Parsons  2004),  tasks  to  ambulance  teams,  fire  brigades  and 
police  forces  in  RoboCup  Rescue  (Nair  et  al.  2002),  tasks 
to  team  members  that  prevent  incursions  of  aircraft  or  boats 
into  launch  impact  zones  (Sycara  et  al.  2005),  different  ob¬ 
servation  targets  to  sensors  in  a  wireless  sensor  network  (for 
example,  for  bird  habitat  sensing)  (Yu  and  Prasanna  2005; 
Howard  and  Viguria  2007),  and  observer  and  manipula¬ 
tor  roles  for  manipulation  tasks  (such  as  in  box  pushing) 
(Gerkey  and  Mataric  2002). 

On-Line  Distributed  Scheduling  and  Control 

Allocate  tasks  (or  processes)  to  machines  (or  processors)  in 
a  distributed  system  to  minimize  latency,  maximize  through¬ 
put  or  optimize  other  objectives,  possibly  considering  con¬ 
straints  among  the  tasks,  such  as  precedence  constraints  or 
assignment  restrictions.  Examples  include  allocating  com¬ 
plex  workflows  with  deadlines  to  CPUs  in  grid  computing 


(Reeves  et  al.  2005)  and  on-line  control  of  environmental 
conditions  in  smart  buildings  (Clearwater  et  al.  1996). 

On-Line  Distributed  Routing 

Allocate  locations  to  agents.  Examples  include  allocating 
accidents  to  ambulances,  incidents  to  police  cars,  mines 
to  autonomous  underwater  vehicles  (for  de-mining)  (Sariel, 
Balch,  and  Stack  2006),  search-and-rescue  locations  to  first 
responders,  customers  to  taxi  cabs,  rocks  to  Mars  rovers  (for 
taking  and  analyzing  rock  probes)  (Tovey  et  al.  2005),  mines 
to  submarines  (for  identification)  (Sariel,  Balch,  and  Stack 
2006),  observation  locations  to  robots  that  map  terrain  (Sim¬ 
mons  et  al.  2000)  and  wings  of  an  art  gallery  to  guard  robots 
(Kalra,  Stentz,  and  Ferguson  2005).  In  this  context,  auc¬ 
tions  have  been  used  on  actual  robots  (Thayer  et  al.  2000; 
Gerkey  and  Mataric  2002;  Zlot  et  al.  2002). 

Testbed:  Multi-Robot  Routing 

The  standard  testbed  of  auction-based  coordination  systems 
is  multi-robot  routing  (Dias  et  al.  2006),  a  special  case  of 
on-line  distributed  routing.  For  multi-robot  routing,  the  bid¬ 
ders  are  the  robots,  and  the  items  up  for  auction  are  tasks  to 
visit  given  targets  (locations).  The  robots  are  identical  and 
know  both  their  own  location  and  the  target  locations.  They 
can  move  and  broadcast  information  without  error.  They 
might  initially  not  know  where  the  obstacles  are  in  the  ter¬ 
rain  but  always  observe  the  ones  in  their  vicinity  without 
error.  For  ease  of  exposition,  we  assume  in  this  overview 
that  the  robots  have  to  visit  all  targets  (and  do  not  need  to 
return  to  their  initial  locations)  with  a  small  sum  of  travel 
distances.  Multi-robot  routing  problems  are  similar  to  trav¬ 
eling  salesperson  problems  and  their  variants  (Lawler  et  al. 
1985),  which  simplifies  their  analysis. 

Auction-Based  Coordination  Systems 

Minimizing  the  sum  of  travel  distances  is  NP-hard  for  multi¬ 
robot  routing  problems  even  if  the  terrain  and  targets  are  ini¬ 
tially  known  and  do  not  change  (Lagoudakis  et  al.  2005). 
Auction-based  coordination  systems  work  as  follows:  Ev¬ 
ery  robot  determines  a  path  with  the  smallest  travel  distance 
to  visit  all  of  the  (unvisited)  targets  that  are  allocated  to  it 
and  starts  to  move  along  the  path.  Thus,  the  robot  does  not 
necessarily  visit  the  targets  in  the  order  in  which  they  were 
allocated  to  it.  Whenever  a  robot  gains  more  information 
about  the  terrain  or  observes  that  the  terrain  has  changed, 
it  shares  this  information  with  the  other  robots.  If  that  in¬ 
formation  increases  the  travel  distance  of  at  least  one  robot 
or  new  targets  are  introduced,  then  all  robots  put  their  (un¬ 
visited)  targets  up  for  auction  for  re-allocation.  Each  robot 
then  bids  in  light  of  the  new  information,  assuming  that  no 
changes  will  be  necessary  in  the  future.  The  auction  closes 
after  a  predetermined  amount  of  time  and  the  robots  are  al¬ 
located  new  targets.  Auction-based  coordination  systems 
based  on  parallel  auctions  allocate  the  targets  in  indepen¬ 
dent  and  simultaneous  single-round  auctions,  one  for  each 
target.  Every  robot  bids  on  all  targets.  They  do  not  take 
synergies  among  targets  into  account,  which  often  results  in 


a  large  sum  of  travel  distances.1  Auction-based  coordina¬ 
tion  systems  based  on  combinatorial  auctions  allocate  the 
targets  in  one  single-round  auction.  Every  robot  bids  on  all 
bundles  (sets)  of  targets.  Hence,  all  synergies  among  tar¬ 
gets  are  taken  into  account,  which  minimizes  the  sum  of 
travel  distances.  However,  an  exponential  (in  the  number 
of  targets)  number  of  bids  must  be  generated,  transmitted, 
and  processed.  The  approximations  necessary  to  guarantee 
real-time  performance  (Berhault  et  al.  2003)  often  interact 
in  unpredictable  ways  and  result  in  complicated  code  that  is 
difficult  to  debug,  maintain  and  integrate  into  robot  archi¬ 
tectures.  Sequential  single-item  (SSI)  auction-based  coordi¬ 
nation  systems  have  recently  emerged  as  a  promising  way 
of  combining  the  advantages  of  auction-based  coordination 
systems  based  on  parallel  auctions  (namely,  their  small  num¬ 
ber  of  bids  and  fast  winner  determination)  and  combinato¬ 
rial  auctions  (namely,  their  small  sum  of  travel  distances) 
(Lagoudakis  et  al.  2005). 

Sequential  Single-Item  (SSI)  Auctions 

The  targets  are  allocated  in  one  multi-round  auction.  During 
each  round,  every  robot  bids  on  each  unallocated  target,  and 
winner  determination  then  allocates  one  additional  target  to 
one  robot.  Every  robot  bids  on  a  target  the  smallest  increase 
in  its  travel  distance  that  would  result  from  it  being  allocated 
the  target  that  it  bids  on  in  addition  to  all  targets  allocated 
to  it  in  previous  rounds  (marginal-cost  bidding)  (Sandholm 
1993).  Winner  determination  determines  the  bid  with  the 
smallest  bid  cost  and  allocates  the  corresponding  target  to 
the  corresponding  robot  (Boutilier,  Goldszmidt,  and  Sabata 
1999;  Fatima  2006). 2  SSI  auction-based  coordination  sys¬ 
tems  take  some  (but  not  all)  synergies  among  targets  into 
account  and  provide  the  following  performance  guarantee  if 
the  terrain  and  targets  are  initially  known  and  do  not  change. 
Some  intuition  for  this  result  can  be  gained  from  interpret¬ 
ing  the  greedy  construction  of  minimum  spanning  trees  as  a 
cooperative  auction  (Lagoudakis  et  al.  2004). 

Theorem  1  (Lagoudakis  et  al.  2005)  The  sum  of  travel 
distances  can  be  a  factor  of  1.5  larger  than  minimal  but  is 
at  most  a  factor  of  two  larger  than  minimal,  whether  each 
robot  calculates  its  travel  distance  exactly  or  uses  the  cheap¬ 
est  insertion  heuristic  (Lawler  et  al.  1985)  to  determine  it 
approximately,  which  results  in  a  polynomial-time  auction 
mechanism. 

SSI  auction-based  coordination  systems  perform  even 
better  experimentally.  It  can  be  shown  that  they  perform 
hill-climbing  in  each  round  by  allocating  one  additional  tar¬ 
get  to  one  robot  so  that  the  sum  of  travel  distances  increases 
the  least.  This  insight  can  be  exploited  to  automatically  de¬ 
termine  how  the  robots  should  bid  for  different  performance 

'Two  targets  have  positive  (negative)  synergy  for  a  robot  if  the 
smallest  travel  distance  for  visiting  both  targets  is  smaller  (larger) 
than  the  sum  of  the  smallest  travel  distances  for  visiting  both  targets 
individually  from  the  current  location  of  the  robot. 

2In  practice,  each  robot  can  determine  the  winning  bid  quickly 
itself  by  listening  to  the  bids  and  identifying  the  one  with  the  small¬ 
est  bid  cost. 


measures,  such  as  having  to  visit  all  targets  with  a  small  task- 
completion  or  flow  time  instead  of  a  small  sum  of  travel  dis¬ 
tances  (Tovey  et  al.  2005).  Furthermore,  every  robot  needs 
to  bid  on  only  one  target  per  round,  namely  the  target  with 
the  smallest  bid  cost,  since  targets  with  larger  bid  costs  can¬ 
not  win.  The  runtime  of  winner  determination  during  each 
round  is  linear  in  the  number  of  submitted  bids,  which  re¬ 
sults  in  the  following  desirable  communication  and  runtime 
complexities  of  winner  determination: 

Proposition  1  (Lagoudakis  et  al.  2005)  The  number  of 
bids  submitted  by  all  robots  during  each  round  and  the  run¬ 
time  of  winner  determination  are  linear  in  the  number  of 
robots  and  independent  of  the  number  of  unallocated  tar¬ 
gets. 

Improving  SSI  Auctions 

Researchers  have  investigated  several  variants  of  SSI  auc¬ 
tions  to  build  SSI  auction-based  coordination  systems  that 
decrease  the  sums  of  travel  distances  while  still  allocating 
tasks  to  robots  in  real  time.  Researchers  have  also  inves¬ 
tigated  how  to  further  improve  the  target  allocation  after¬ 
wards,  for  example,  using  task  swaps  among  robots  (Dias 
and  Stentz  2000;  Zheng  and  Koenig  2009). 

SSI  Auctions  with  Rollouts 

Everything  is  the  same  as  for  SSI  auction-based  coordina¬ 
tion  systems,  except  that  every  robot  now  bids  on  a  target  the 
sum  of  travel  distances  of  the  complete  target  allocation  that 
would  result  from  it  being  allocated  the  target  that  it  bids  on 
in  addition  to  all  targets  allocated  to  it  in  previous  rounds,  all 
other  robots  being  allocated  the  targets  allocated  to  them  in 
previous  rounds,  and  then  hill-climbing  completing  this  par¬ 
tial  target  allocation  to  a  complete  target  allocation  (Zheng, 
Koenig,  and  Tovey  2006).  The  bid  costs  of  the  robots  are 
now  more  informed  since  they  are  based  on  complete  rather 
than  partial  target  allocations,  which  takes  more  synergies 
among  targets  into  account  and  makes  hill-climbing  less  my¬ 
opic.  This  is  most  helpful  in  the  early  rounds  where  the  tar¬ 
get  allocations  are  far  from  being  complete.  Proposition  1 
continues  to  hold  trivially.  However,  the  number  of  rounds 
increases  since  each  round  of  the  main  SSI  auction  is  now 
preceeded  by  the  rounds  of  the  SSI  auctions  for  the  corre¬ 
sponding  (parallel)  rollouts. 

SSI  Auctions  with  Bundle  Bids 

Everything  is  the  same  as  for  SSI  auction-based  coordina¬ 
tion  systems,  except  that  every  robot  now  bids  on  each  bun¬ 
dle  of  at  most  k  unallocated  targets,  and  winner  determina¬ 
tion  then  allocates  k  additional  targets  to  one  or  more  robots, 
making  SSI  auctions  with  bundle  bids  the  same  as  standard 
SSI  auctions  if  k  =  1  and  the  same  as  combinatorial  auc¬ 
tions  if  k  is  large  (Koenig  et  al.  2007).  SSI  auction-based 
coordination  systems  with  bundle  bids  allocate  k  additional 
targets  to  one  or  more  robots  in  each  round  so  that  the  sum  of 
travel  distances  increases  the  least,  which  takes  more  syner¬ 
gies  among  targets  into  account  and  makes  hill-climbing  less 
myopic.  It  can  be  shown  that  every  robot  needs  to  bid  on 
only  a  constant  number  of  bundles  per  round  since  the  other 


bundles  cannot  win.  These  bundles  can  be  determined  au¬ 
tomatically.  For  example,  every  robot  needs  to  bid  on  three 
bundles  per  round  if  k  =  2,  namely  the  single-target  bundles 
with  the  two  lowest  bid  costs  and  the  double-target  bundle 
with  the  lowest  bid  cost,  and  only  seven  bundles  per  round 
if  k  =  3,  which  can  reduce  the  number  of  bids  by  several 
orders  of  magnitude.  For  example,  the  number  of  bids  sub¬ 
mitted  by  every  robot  per  round  for  twenty  unallocated  tar¬ 
gets  and  k  =  6  is  only  105  instead  of  60,459  (Koenig  et  al. 
2007).  The  winner  determination  procedure  can  be  automat¬ 
ically  determined  and,  to  decrease  runtime,  be  compiled  into 
compact  program  code  (Daniel  and  Koenig  2009).  Propo¬ 
sition  1  continues  to  hold  for  fixed  k  (although  the  proof 
is  nontrivial),  which  makes  SSI  auctions  with  bundle  bids 
an  attractive  cross  between  SSI  and  combinatorial  auctions 
(Koenig  et  al.  2007). 

SSI  Auctions  with  Regret  Clearing 

Everything  is  the  same  as  for  SSI  auction-based  coordina¬ 
tion  systems,  except  that  winner  determination  now  allocates 
the  target  with  the  largest  regret  to  the  robot  whose  bid  cost 
on  it  is  smallest,  where  the  regret  of  a  target  is  the  difference 
of  the  second-smallest  and  smallest  bid  cost  on  it  (Koenig  et 
al.  2008).  If  the  terrain  and  targets  are  initially  known  and 
do  not  change,  SSI  auction-based  coordination  systems  with 
regret  clearing  provide  the  following  performance  guaran¬ 
tee,  which  could  be  worse  than  that  of  Theorem  1. 

Theorem  2  (Koenig  et  al.  2008)  The  sum  of  travel  dis¬ 
tances  can  be  a  factor  of  three  larger  than  minimal  but  is 
at  most  a  factor  of  twice  the  number  of  targets  larger  than 
minimal,  whether  each  robot  calculates  its  travel  distance 
exactly  or  uses  the  cheapest  insertion  heuristic  to  determine 
it  approximately,  which  results  in  a  polynomial-time  auction 
mechanism. 

However,  SSI  auction-based  coordination  systems  with 
regret  clearing  can  perform  much  better  experimentally  than 
standard  SSI  auction-based  coordination  systems,  which  can 
be  explained  as  follows:  Later  allocations  of  targets  to  robots 
are  typically  more  informed  than  earlier  ones  since  the  tar¬ 
get  allocations  in  earlier  rounds  are  far  from  being  com¬ 
plete.  If  a  target  is  allocated  to  a  robot  in  the  current  round 
then  one  wants  to  ensure  that,  if  this  allocation  were  post¬ 
poned,  the  same  allocation  would  be  made  in  later  rounds. 
SSI  auction-based  coordination  systems  with  regret  clearing 
achieve  this  objective  by  no  longer  performing  hill-climbing 
in  each  round  but  rather  maximizing  the  difference  of  the 
second-smallest  and  smallest  team  costs  that  would  result 
from  the  second-best  and  best  robot,  respectively,  being  al¬ 
locating  one  additional  target.  The  communication  and  run¬ 
time  complexities  of  winner  determination  remain  polyno¬ 
mial  but  Proposition  1  no  longer  holds  since  every  robot  now 
needs  to  bid  on  all  unallocated  targets  per  round. 

Extensions 

Auction-based  coordination  systems  have  also  been  success¬ 
fully  applied  to  several  generalizations  of  multi-robot  rout¬ 
ing  problems,  for  example,  where  a  team  of  robots  has  to 
visit  a  set  of  given  targets  with  linear  decreasing  rewards 


over  time,  such  as  required  for  the  delivery  of  goods  to  res¬ 
cue  sites  after  disasters.  The  robots  have  to  visit  a  subset 
of  targets  so  as  to  maximize  the  surplus,  which  is  defined 
to  be  the  sum  of  the  rewards  of  the  visited  targets  minus 
the  sum  of  travel  costs.  Auction-based  coordination  sys¬ 
tems  are  able  to  solve  these  NP-hard  problems  in  seconds 
and  with  a  surplus  that  is  comparable  to  the  surplus  found 
by  a  mixed  integer  program  with  a  12  hour  time  limit  (Ekici, 
Keskinocak,  and  Koenig  2009).  Auction-based  coordination 
systems  have  also  been  applied  to  multi-robot  routing  prob¬ 
lems  where  a  team  of  robots  has  to  visit  a  set  of  given  targets 
with  given  priorities  during  given  time  windows  that  do  not 
overlap,  such  as  required  for  planetary  exploration.  Again, 
auction-based  coordination  systems  are  able  to  solve  these 
NP-hard  problems  in  seconds  and  with  good  team  perfor¬ 
mance  (Melvin  et  al.  2007).  Current  work  includes  apply¬ 
ing  SSI  auction-based  coordination  systems  to  multi-robot 
routing  problems  where  robots  need  to  visit  some  targets  si¬ 
multaneously,  such  as  required  for  moving  large  obstacles 
out  of  the  way  cooperatively.  Every  robot  now  needs  to  bid 
on  each  pair  of  time  slot  and  unallocated  target.  The  result¬ 
ing  function  maps  pairs  of  time  slots  and  unallocated  targets 
to  bid  costs  but  is  typically  more  compact  and  approximated 
more  easily  if  it  is  expressed  as  a  function  that  maps  unallo¬ 
cated  targets  to  reaction  functions,  where  a  reaction  function 
maps  time  slots  to  bid  costs  (Zheng  and  Koenig  2008). 

Future  Work 

There  are  many  alternative  approaches  for  building  coordi¬ 
nation  systems,  both  centralized  (for  example,  mixed  integer 
programming)  and  decentralized  (for  example,  distributed 
constraint  optimization  or  token  passing).  More  work  needs 
to  be  done  on  determining  when  to  use  which  approach  since 
the  strengths  and  weaknesses  of  the  individual  approaches 
are  not  yet  well  understood  and  only  a  few  experimental 
comparisons  exist  (Xu  et  al.  2006).  More  work  needs  to  be 
done  on  developing  auction-based  coordination  systems  that 
better  exploit  the  local  (private)  information  of  the  agents 
and  auction-based  coordination  systems  for  heterogeneous 
agents.  More  generally,  more  work  needs  to  done  on  ap¬ 
plying  auction-based  coordination  systems  in  more  complex 
application  domains  than  has  been  done  so  far. 

•  Example  1:  Consider  a  heterogeneous  team  of  two  dif¬ 
ferent  kinds  of  agents,  namely  general  agents  X  that  can 
perform  both  tasks  A  and  B,  and  specialized  agents  Y  that 
can  perform  only  task  A.  Then,  it  might  not  be  a  good 
idea  to  let  the  myopically  best  agent  execute  a  task  (which 
is  what  SSI  auction-based  coordination  systems  do  cur¬ 
rently).  For  example,  if  two  agents  X  and  Y  are  available 
and  agent  X  is  assigned  to  execute  task  A,  then  no  agent  is 
available  to  execute  an  arriving  task  B.  On  the  other  hand, 
if  agent  Y  is  assigned  to  execute  task  A,  then  agent  X  is 
still  available  to  execute  an  arriving  task  A  or  B.  Thus, 
the  agents  need  to  predict  the  future  to  achieve  a  small 
team  cost,  perhaps  using  methods  from  machine  learning 
(Schneider  et  al.  2005). 

•  Example  2:  Consider  tasks  that  involve  time-consuming 
planning  or  scheduling  to  determine  the  bid  costs  of  the 


agents.  Then,  the  agents  can  calculate  only  a  limited  num¬ 
ber  of  bid  costs  and  first  need  to  determine  which  bid  costs 
to  calculate.  Thus,  they  need  to  predict  the  bid  costs  be¬ 
fore  calculating  them,  perhaps  again  using  methods  from 
machine  learning  (Busquets  and  Simmons  2006). 

Finally,  more  work  needs  to  be  done  on  making  auction- 
based  coordination  systems  robust  against  error  (Sariel, 
Balch,  and  Erdogan  2006;  Nanjanath  and  Gini  2008),  for 
example,  to  ensure  that  each  target  gets  visited  even  when 
robots  fail  or  leave  the  communication  range  of  other  robots. 
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